Benchmark Tests For The Darpa Spoken Language Program
نویسندگان
چکیده
This paper documents benchmark tests implemented within the DARPA Spoken Language Program during the period November, 1992 January, 1993. Tests were conducted using the Wall Street Journal-based Continuous Speech Recognition (WSJ-CSR) corpus and the Air Travel Information System (ATIS) corpus collected by the Multi-site ATIS Data COllection Working (MADCOW) Group. The WSJ-CSR tests consist of tests of large vocabulary (lexicons of 5,000 to more than 20,000 words) continuous speech recognition systems. The ATIS tests consist of tests of (1) ATIS-domain spontaneous speech (lexicons typically less than 2,000 words), (2) natural language understanding, and (3) spoken language understanding. These tests were reported on and discussed in detail at the Spoken Language Systems Technology Workshop held at the Massachusetts Institute of Technology, January 20-22, 1993.
منابع مشابه
NATIONAL INSTITUTE OF STANDARDS AND TECHNOLOGY (NIST) (Formerly National Bureau of Standards)
Our objectives in the DARPA Spoken Language Program are: (1) to provide a central role in speech database design, development and distribution; and (2) to design, coordinate implementation of, and analyze results of performance tests. (3) Prepared the Prototype CD-ROM Version of the DARPA TIMIT Acoustic-Phonetic Speech Database. This CD-ROM contains the data for 420 talkers (4200 sentence utter...
متن کاملDARPA February 1992 ATIS Benchmark Test Results
This paper documents the third in a series of Benchmark Tests for the DARPA Air Travel Information System (ATIS) common task domain. The first results in this series were reported at the June 1990 Speech and Natural Language Workshop [1], and the second at the February 1991 Speech and Natural Language Workshop [2]. The February 1992 Benchmark Tests include: (1) ATIS domain spontaneous speech re...
متن کاملNIST-DARPA Interagency Agreement: Spoken Language Program
1. To coordinate the design, development and distribution of speech and natural language corpora for the DARPA Spoken Language research community. 2. To design, coordinate implementation, and analyze results, of performance assessment "benchmark tests" for DARPA's speech recognition and spoken language understanding systems. 1. Completed production of the six-CD-ROM-set for ATIS0, and made this...
متن کامل1993 Benchmark Tests for the ARPA Spoken Language Program
This paper reports results obtained in benchmark tests conducted within the ARPA Spoken Language program in November and December of 1993. In addition to ARPA contractors, participants included a number of %olunteers", including foreign participants from Canada, France, Germany, and the United Kingdom. The body of the paper is limited to an outline of the structure of the tests and presents hig...
متن کاملA Real-Time Spoken-Language System for Interactive Problem Solving
SRI has developed a spoken language system to retrieve air travel planning information. Progress can be measured by comparing DARPA benchmark results in February 1992 and November 1992. Between February 1992 and November 1992, for all utterances tested, SRJ's word error rate in the ATIS speech recognition test improved from 11.0% to 9.1%. Weighted utterance error improved from 31.1% to 23.6% in...
متن کامل